<!DOCTYPE html>


<html lang="zh-CN">


<head>
  <meta charset="utf-8" />
    
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1" />
  <title>
    kubernetes 常见问题总结.md |  
  </title>
  <meta name="generator" content="hexo-theme-ayer">
  
  <link rel="shortcut icon" href="/favicon.ico" />
  
  
<link rel="stylesheet" href="/dist/main.css">

  
<link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/Shen-Yu/cdn/css/remixicon.min.css">

  
<link rel="stylesheet" href="/css/custom.css">

  
  
<script src="https://cdn.jsdelivr.net/npm/pace-js@1.0.2/pace.min.js"></script>

  
  

  

</head>

</html>

<body>
  <div id="app">
    
      
    <main class="content on">
      <section class="outer">
  <article
  id="post-k8s/kubernetes 常见问题总结"
  class="article article-type-post"
  itemscope
  itemprop="blogPost"
  data-scroll-reveal
>
  <div class="article-inner">
    
    <header class="article-header">
       
<h1 class="article-title sea-center" style="border-left:0" itemprop="name">
  kubernetes 常见问题总结.md
</h1>
 

    </header>
     
    <div class="article-meta">
      <a href="/2020/11/11/k8s/kubernetes%20%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98%E6%80%BB%E7%BB%93/" class="article-date">
  <time datetime="2020-11-10T16:00:00.000Z" itemprop="datePublished">2020-11-11</time>
</a> 
  <div class="article-category">
    <a class="article-category-link" href="/categories/k8s/">k8s</a>
  </div>
  
<div class="word_count">
    <span class="post-time">
        <span class="post-meta-item-icon">
            <i class="ri-quill-pen-line"></i>
            <span class="post-meta-item-text"> 字数统计:</span>
            <span class="post-count">3.5k</span>
        </span>
    </span>

    <span class="post-time">
        &nbsp; | &nbsp;
        <span class="post-meta-item-icon">
            <i class="ri-book-open-line"></i>
            <span class="post-meta-item-text"> 阅读时长≈</span>
            <span class="post-count">18 分钟</span>
        </span>
    </span>
</div>
 
    </div>
      
    <div class="tocbot"></div>




  
    <div class="article-entry" itemprop="articleBody">
       
  <h1 id="kubernetes-常见问题总结"><a href="#kubernetes-常见问题总结" class="headerlink" title="kubernetes 常见问题总结"></a>kubernetes 常见问题总结</h1><p>如何删除不一致状态下的 rc,deployment,service</p>
<p>在某些情况下,经常发现 kubectl 进程挂起现象,然后在 get 时候发现删了一半,而另外的删除不了</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[root@k8s-master ~]# kubectl get -f fluentd-elasticsearch&#x2F;  </span><br><span class="line">NAME DESIRED CURRENT READY AGE  </span><br><span class="line">rc&#x2F;elasticsearch-logging-v1 0 2 2 15h  </span><br><span class="line">  </span><br><span class="line">NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE  </span><br><span class="line">deploy&#x2F;kibana-logging 0 1 1 1 15h  </span><br><span class="line">Error from server (NotFound): services &quot;elasticsearch-logging&quot; not found  </span><br><span class="line">Error from server (NotFound): daemonsets.extensions &quot;fluentd-es-v1.22&quot; not found  </span><br><span class="line">Error from server (NotFound): services &quot;kibana-logging&quot; not found</span><br></pre></td></tr></table></figure>



<p>删除这些 deployment,service 或者 rc 命令如下:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">kubectl delete deployment kibana-logging -n kube-system --cascade&#x3D;false  </span><br><span class="line">  </span><br><span class="line">kubectl delete deployment kibana-logging -n kube-system  --ignore-not-found  </span><br><span class="line">  </span><br><span class="line">delete rc elasticsearch-logging-v1 -n kube-system --force now --grace-period&#x3D;0  </span><br><span class="line">1|2删除不了后如何重置etcd</span><br></pre></td></tr></table></figure>



<p>删除不了后如何重置 etcd</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">rm -rf &#x2F;var&#x2F;lib&#x2F;etcd&#x2F;*</span><br></pre></td></tr></table></figure>

<p>删除后重新 reboot master 结点。</p>
<p>reset etcd 后需要重新设置网络</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">etcdctl mk &#x2F;atomic.io&#x2F;network&#x2F;config &#39;&#123; &quot;Network&quot;: &quot;192.168.0.0&#x2F;16&quot; &#125;&#39;</span><br></pre></td></tr></table></figure>



<p>启动 apiserver 失败</p>
<p>每次启动都是报如下问题：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">start request repeated too quickly for kube-apiserver.service</span><br></pre></td></tr></table></figure>



<p>但其实不是启动频率问题,需要查看, /var/log/messages,在我的情况中是因为开启    ServiceAccount 后找不到 ca.crt 等文件,导致启动出错</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">May 21 07:56:41 k8s-master kube-apiserver: Flag --port has been deprecated, see --insecure-port instead.  </span><br><span class="line">May 21 07:56:41 k8s-master kube-apiserver: F0521 07:56:41.692480 4299 universal_validation.go:104] Validate server run options failed: unable to load client CA file: open &#x2F;var&#x2F;run&#x2F;kubernetes&#x2F;ca.crt: no such file or directory  </span><br><span class="line">May 21 07:56:41 k8s-master systemd: kube-apiserver.service: main process exited, code&#x3D;exited, status&#x3D;255&#x2F;n&#x2F;a  </span><br><span class="line">May 21 07:56:41 k8s-master systemd: Failed to start Kubernetes API Server.  </span><br><span class="line">May 21 07:56:41 k8s-master systemd: Unit kube-apiserver.service entered failed state.  </span><br><span class="line">May 21 07:56:41 k8s-master systemd: kube-apiserver.service failed.  </span><br><span class="line">May 21 07:56:41 k8s-master systemd: kube-apiserver.service holdoff time over, scheduling restart.  </span><br><span class="line">May 21 07:56:41 k8s-master systemd: start request repeated too quickly for kube-apiserver.service  </span><br><span class="line">May 21 07:56:41 k8s-master systemd: Failed to start Kubernetes API Server.</span><br></pre></td></tr></table></figure>



<p>在部署 fluentd 等日志组件的时候,很多问题都是因为需要开启 ServiceAccount 选项需要配置安全导致,所以说到底还是需要配置好 ServiceAccount.</p>
<p>出现 Permission denied 情况</p>
<p><strong>在配置 fluentd 时候出现cannot create /var/log/fluentd.log: Permission denied 错误,这是因为没有关掉 SElinux 安全导致。</strong>可以在 /etc/selinux/config 中将 SELINUX=enforcing 设置成 disabled,然后 reboot</p>
<p>基于 ServiceAccount 的配置</p>
<p>首先生成各种需要的 keys,k8s-master 需替换成 master 的主机名.</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">openssl genrsa -out ca.key 2048  </span><br><span class="line">openssl req -x509 -new -nodes -key ca.key -subj &quot;&#x2F;CN&#x3D;k8s-master&quot; -days 10000 -out ca.crt  </span><br><span class="line">openssl genrsa -out server.key 2048  </span><br><span class="line">  </span><br><span class="line">echo subjectAltName&#x3D;IP:10.254.0.1 &gt; extfile.cnf  </span><br><span class="line">  </span><br><span class="line">#ip由下述命令决定  </span><br><span class="line">  </span><br><span class="line">#kubectl get services --all-namespaces |grep &#39;default&#39;|grep &#39;kubernetes&#39;|grep &#39;443&#39;|awk &#39;&#123;print $3&#125;&#39;  </span><br><span class="line">  </span><br><span class="line">  </span><br><span class="line">openssl req -new -key server.key -subj &quot;&#x2F;CN&#x3D;k8s-master&quot; -out server.csr  </span><br><span class="line">  </span><br><span class="line">openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -extfile extfile.cnf -out server.crt -days 10000</span><br></pre></td></tr></table></figure>



<p>如果修改 /etc/kubernetes/apiserver 的配置文件参数的话,通过 systemctl start kube-apiserver 启动失败,出错信息为:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">Validate server run options failed: unable to load client CA file: open &#x2F;root&#x2F;keys&#x2F;ca.crt: permission denied</span><br></pre></td></tr></table></figure>



<p>但可以通过命令行启动 API Server</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">&#x2F;usr&#x2F;bin&#x2F;kube-apiserver --logtostderr&#x3D;true --v&#x3D;0 --etcd-servers&#x3D;http:&#x2F;&#x2F;k8s-master:2379 --address&#x3D;0.0.0.0 --port&#x3D;8080 --kubelet-port&#x3D;10250 --allow-privileged&#x3D;true --service-cluster-ip-range&#x3D;10.254.0.0&#x2F;16 --admission-control&#x3D;ServiceAccount --insecure-bind-address&#x3D;0.0.0.0 --client-ca-file&#x3D;&#x2F;root&#x2F;keys&#x2F;ca.crt --tls-cert-file&#x3D;&#x2F;root&#x2F;keys&#x2F;server.crt --tls-private-key-file&#x3D;&#x2F;root&#x2F;keys&#x2F;server.key --basic-auth-file&#x3D;&#x2F;root&#x2F;keys&#x2F;basic_auth.csv --secure-port&#x3D;443 &amp;&gt;&gt; &#x2F;var&#x2F;log&#x2F;kubernetes&#x2F;kube-apiserver.log &amp;</span><br></pre></td></tr></table></figure>



<p>命令行启动 Controller-manager</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">&#x2F;usr&#x2F;bin&#x2F;kube-controller-manager --logtostderr&#x3D;true --v&#x3D;0 --master&#x3D;http:&#x2F;&#x2F;k8s-master:8080 --root-ca-file&#x3D;&#x2F;root&#x2F;keys&#x2F;ca.crt --service-account-private-key-file&#x3D;&#x2F;root&#x2F;keys&#x2F;server.key &amp; &gt;&gt;&#x2F;var&#x2F;log&#x2F;kubernetes&#x2F;kube-controller-manage.log</span><br></pre></td></tr></table></figure>





<p>ETCD 启动不起来-问题&lt;1&gt;</p>
<p>etcd 是 kubernetes 集群的 zookeeper 进程，几乎所有的 service 都依赖于 etcd  的启动，比如 flanneld,apiserver,docker…..</p>
<p><strong>在启动 etcd 是报错日志如下：</strong></p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br></pre></td><td class="code"><pre><span class="line">May 24 13:39:09 k8s-master systemd: Stopped Flanneld overlay address etcd agent.  </span><br><span class="line">May 24 13:39:28 k8s-master systemd: Starting Etcd Server...  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS&#x3D;http:&#x2F;&#x2F;etcd:2379,http:&#x2F;&#x2F;etcd:4001  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: recognized environment variable ETCD_NAME, but unused: shadowed by corresponding flag  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: recognized environment variable ETCD_DATA_DIR, but unused: shadowed by corresponding flag  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: recognized environment variable ETCD_LISTEN_CLIENT_URLS, but unused: shadowed by corresponding flag  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: etcd Version: 3.1.3  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: Git SHA: 21fdcc6  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: Go Version: go1.7.4  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: Go OS&#x2F;Arch: linux&#x2F;amd64  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: setting maximum number of CPUs to 1, total number of available CPUs is 1  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: the server is already initialized as member before, starting as etcd member...  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: listening for peers on http:&#x2F;&#x2F;localhost:2380  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: listening for client requests on 0.0.0.0:2379  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: listening for client requests on 0.0.0.0:4001  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: recovered store from snapshot at index 140014  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: name &#x3D; master  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: data dir &#x3D; &#x2F;var&#x2F;lib&#x2F;etcd&#x2F;default.etcd  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: member dir &#x3D; &#x2F;var&#x2F;lib&#x2F;etcd&#x2F;default.etcd&#x2F;member  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: heartbeat &#x3D; 100ms  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: election &#x3D; 1000ms  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: snapshot count &#x3D; 10000  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: advertise client URLs &#x3D; http:&#x2F;&#x2F;etcd:2379,http:&#x2F;&#x2F;etcd:4001  </span><br><span class="line">May 24 13:39:28 k8s-master etcd: ignored file 0000000000000001-0000000000012700.wal.broken in wal  </span><br><span class="line">May 24 13:39:29 k8s-master etcd: restarting member 8e9e05c52164694d in cluster cdf818194e3a8c32 at commit index 148905  </span><br><span class="line">May 24 13:39:29 k8s-master etcd: 8e9e05c52164694d became follower at term 12  </span><br><span class="line">May 24 13:39:29 k8s-master etcd: newRaft 8e9e05c52164694d [peers: [8e9e05c52164694d], term: 12, commit: 148905, applied: 140014, lastindex: 148905, lastterm: 12]  </span><br><span class="line">May 24 13:39:29 k8s-master etcd: enabled capabilities for version 3.1  </span><br><span class="line">May 24 13:39:29 k8s-master etcd: added member 8e9e05c52164694d [http:&#x2F;&#x2F;localhost:2380] to cluster cdf818194e3a8c32 from store  </span><br><span class="line">May 24 13:39:29 k8s-master etcd: set the cluster version to 3.1 from store  </span><br><span class="line">May 24 13:39:29 k8s-master etcd: starting server... [version: 3.1.3, cluster version: 3.1]  </span><br><span class="line">May 24 13:39:29 k8s-master etcd: raft save state and entries error: open &#x2F;var&#x2F;lib&#x2F;etcd&#x2F;default.etcd&#x2F;member&#x2F;wal&#x2F;0.tmp: is a directory  </span><br><span class="line">May 24 13:39:29 k8s-master systemd: etcd.service: main process exited, code&#x3D;exited, status&#x3D;1&#x2F;FAILURE  </span><br><span class="line">May 24 13:39:29 k8s-master systemd: Failed to start Etcd Server.  </span><br><span class="line">May 24 13:39:29 k8s-master systemd: Unit etcd.service entered failed state.  </span><br><span class="line">May 24 13:39:29 k8s-master systemd: etcd.service failed.  </span><br><span class="line">May 24 13:39:29 k8s-master systemd: etcd.service holdoff time over, scheduling restart.</span><br></pre></td></tr></table></figure>



<p><strong>核心语句：</strong></p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">raft save state and entries error: open &#x2F;var&#x2F;lib&#x2F;etcd&#x2F;default.etcd&#x2F;member&#x2F;wal&#x2F;0.tmp: is a directory</span><br></pre></td></tr></table></figure>

<p>进入相关目录，删除 0.tmp,然后就可以启动啦!</p>
<p>ETCD启动不起来-超时问题&lt;2&gt;</p>
<p>问题背景：当前部署了 3 个 etcd 节点，突然有一天 3 台集群全部停电宕机了。重新启动之后发现 K8S 集群是可以正常使用的，但是检查了一遍组件之后，发现有一个节点的 etcd 启动不了。</p>
<p>经过一遍探查，发现时间不准确，通过以下命令 <strong>ntpdate ntp.aliyun.com</strong> 重新将时间调整正确，重新启动 etcd，发现还是起不来，报错如下：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br></pre></td><td class="code"><pre><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: etcd Version: 3.3.13  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: Git SHA: 98d3084  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: Go Version: go1.10.8  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: Go OS&#x2F;Arch: linux&#x2F;amd64  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: setting maximum number of CPUs to 4, total number of available CPUs is 4  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: the server is already initialized as member before, starting as etcd member  </span><br><span class="line">...  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: peerTLS: cert &#x3D; &#x2F;opt&#x2F;etcd&#x2F;ssl&#x2F;server.pem, key &#x3D; &#x2F;opt&#x2F;etcd&#x2F;ssl&#x2F;server-key.pe  </span><br><span class="line">m, ca &#x3D; , trusted-ca &#x3D; &#x2F;opt&#x2F;etcd&#x2F;ssl&#x2F;ca.pem, client-cert-auth &#x3D; false, crl-file &#x3D;  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: listening for peers on https:&#x2F;&#x2F;192.168.25.226:2380  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: The scheme of client url http:&#x2F;&#x2F;127.0.0.1:2379 is HTTP while peer key&#x2F;cert  </span><br><span class="line">files are presented. Ignored key&#x2F;cert files.  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: listening for client requests on 127.0.0.1:2379  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: listening for client requests on 192.168.25.226:2379  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3248]: member 9c166b8b7cb6ecb8 has already been bootstrapped  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 systemd[1]: etcd.service: main process exited, code&#x3D;exited, status&#x3D;1&#x2F;FAILURE  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 systemd[1]: Failed to start Etcd Server.  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 systemd[1]: Unit etcd.service entered failed state.  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 systemd[1]: etcd.service failed.  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 systemd[1]: etcd.service failed.  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 systemd[1]: etcd.service holdoff time over, scheduling restart.  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 systemd[1]: Starting Etcd Server...  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3258]: recognized environment variable ETCD_NAME, but unused: shadowed by correspo  </span><br><span class="line">nding flag  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3258]: recognized environment variable ETCD_DATA_DIR, but unused: shadowed by corr  </span><br><span class="line">esponding flag  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3258]: recognized environment variable ETCD_LISTEN_PEER_URLS, but unused: shadowed  </span><br><span class="line"> by corresponding flag  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3258]: recognized environment variable ETCD_LISTEN_CLIENT_URLS, but unused: shadow  </span><br><span class="line">ed by corresponding flag  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3258]: recognized environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS, but unuse  </span><br><span class="line">d: shadowed by corresponding flag  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3258]: recognized environment variable ETCD_ADVERTISE_CLIENT_URLS, but unused: sha  </span><br><span class="line">dowed by corresponding flag  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3258]: recognized environment variable ETCD_INITIAL_CLUSTER, but unused: shadowed  </span><br><span class="line">by corresponding flag  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3258]: recognized environment variable ETCD_INITIAL_CLUSTER_TOKEN, but unused: sha  </span><br><span class="line">dowed by corresponding flag  </span><br><span class="line">Mar 05 14:27:15 k8s-node2 etcd[3258]: recognized environment variable ETCD_INITIAL_CLUSTER_STATE, but unused: sha  </span><br><span class="line">dowed by corresponding flag</span><br></pre></td></tr></table></figure>



<p><strong>解决方法：</strong></p>
<p>检查日志发现并没有特别明显的错误，根据经验来讲，etcd 节点坏掉一个其实对集群没有大的影响，这时集群已经可以正常使用了，但是这个坏掉的 etcd 节点并没有启动，解决方法如下：</p>
<ol>
<li><p>进入 etcd 的数据存储目录进行备份</p>
<p>备份原有数据：</p>
<p>cd /var/lib/etcd/default.etcd/member/</p>
<p>cp *  /data/bak/</p>
</li>
<li><p>删除这个目录下的所有数据文件</p>
<p>rm -rf /var/lib/etcd/default.etcd/member/*</p>
</li>
<li><p>停止另外两台 etcd 节点，因为 etcd 节点启动时需要所有节点一起启动，启动成功后即可使用。</p>
</li>
</ol>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">master 节点  </span><br><span class="line">systemctl stop etcd  </span><br><span class="line">systemctl restart etcd  </span><br><span class="line">  </span><br><span class="line">node1 节点  </span><br><span class="line">systemctl stop etcd  </span><br><span class="line">systemctl restart etcd  </span><br><span class="line">  </span><br><span class="line">node2 节点  </span><br><span class="line">systemctl stop etcd  </span><br><span class="line">systemctl restart etcd</span><br></pre></td></tr></table></figure>



<p>CentOS下配置主机互信</p>
<ul>
<li>在每台服务器需要建立主机互信的用户名执行以下命令生成公钥/密钥，默认回车即可</li>
</ul>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ssh-keygen -t rsa</span><br></pre></td></tr></table></figure>

<p>可以看到生成个公钥的文件</p>
<ul>
<li>互传公钥，第一次需要输入密码，之后就OK了</li>
</ul>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ssh-copy-id -i &#x2F;root&#x2F;.ssh&#x2F;id_rsa.pub root@192.168.199.132 (-p 2222)</span><br></pre></td></tr></table></figure>

<p>-p 端口 默认端口不加 -p，如果更改过端口，就得加上 -p . 可以看到是在 .ssh/ 下生成了个 authorized_keys 的文件，记录了能登陆这台服务器的其他服务器的公钥</p>
<ul>
<li>测试看是否能登陆</li>
</ul>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ssh 192.168.199.132 (-p 2222)</span><br></pre></td></tr></table></figure>



<p>CentOS 主机名的修改</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">hostnamectl set-hostname k8s-master1</span><br></pre></td></tr></table></figure>



<p>Virtualbox 实现 CentOS 复制和粘贴功能</p>
<p>如果不安装或者不输出，可以将 update 修改成 install 再运行</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">yum install update  </span><br><span class="line">yum update kernel  </span><br><span class="line">yum update kernel-devel  </span><br><span class="line">yum install kernel-headers  </span><br><span class="line">yum install gcc  </span><br><span class="line">yum install gcc make</span><br></pre></td></tr></table></figure>



<p>运行完后 sh VBoxLinuxAdditions.run</p>
<p>删除 Pod 一直处于 Terminating 状态</p>
<p>可以通过下面命令强制删除</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">kubectl delete pod NAME --grace-period&#x3D;0 --force</span><br></pre></td></tr></table></figure>



<hr>
<p>删除 namespace 一直处于 Terminating 状态</p>
<p>可以通过以下脚本强制删除</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line">[root@k8s-master1 k8s]# cat delete-ns.sh  </span><br><span class="line">#!&#x2F;bin&#x2F;bash  </span><br><span class="line">set -e  </span><br><span class="line">  </span><br><span class="line">useage()&#123;  </span><br><span class="line">    echo &quot;useage:&quot;  </span><br><span class="line">    echo &quot; delns.sh NAMESPACE&quot;  </span><br><span class="line">&#125;  </span><br><span class="line">  </span><br><span class="line">if [ $# -lt 1 ];then  </span><br><span class="line">    useage  </span><br><span class="line">    exit  </span><br><span class="line">fi  </span><br><span class="line">  </span><br><span class="line">NAMESPACE&#x3D;$1  </span><br><span class="line">JSONFILE&#x3D;$&#123;NAMESPACE&#125;.json  </span><br><span class="line">kubectl get ns &quot;$&#123;NAMESPACE&#125;&quot; -o json &gt; &quot;$&#123;JSONFILE&#125;&quot;  </span><br><span class="line">vi &quot;$&#123;JSONFILE&#125;&quot;  </span><br><span class="line">curl -k -H &quot;Content-Type: application&#x2F;json&quot; -X PUT --data-binary @&quot;$&#123;JSONFLE&#125;&quot; \  </span><br><span class="line">    http:&#x2F;&#x2F;127.0.0.1:8001&#x2F;api&#x2F;v1&#x2F;namespaces&#x2F;&quot;$&#123;NAMESPACE&#125;&quot;&#x2F;finalize</span><br></pre></td></tr></table></figure>



<p>容器包含有效的 CPU/内存 requests 且没有指定 limits 可能会出现什么问题？</p>
<p>下面我们创建一个对应的容器，该容器只有 requests 设定，但是没有 limits 设定，</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">- name: busybox-cnt02  </span><br><span class="line">    image: busybox  </span><br><span class="line">    command: [&quot;&#x2F;bin&#x2F;sh&quot;]  </span><br><span class="line">    args: [&quot;-c&quot;, &quot;while true; do echo hello from cnt02; sleep 10;done&quot;]  </span><br><span class="line">    resources:  </span><br><span class="line">      requests:  </span><br><span class="line">        memory: &quot;100Mi&quot;  </span><br><span class="line">        cpu: &quot;100m&quot;</span><br></pre></td></tr></table></figure>



<p><strong>这个容器创建出来会有什么问题呢？</strong><br>其实对于正常的环境来说没有什么问题，但是对于资源型 pod 来说，如果有的容器没有设定 limit 限制，资源会被其他的 pod 抢占走，可能会造成容器应用失败的情况。可以通过 limitrange 策略来去匹配，让 pod 自动设定，前提是要提前配置好limitrange 规则。</p>
<p>- END -</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure>


<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure>


<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"></span><br><span class="line"> 推荐阅读   </span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure>


<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure>

<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure>


<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"></span><br><span class="line">[DevOps 工程师该懂些什么？](http:&#x2F;&#x2F;mp.weixin.qq.com&#x2F;s?__biz&#x3D;MzAwNTM5Njk3Mw&#x3D;&#x3D;&amp;mid&#x3D;2247494357&amp;idx&#x3D;1&amp;sn&#x3D;071829bb4c381ce2c286977c6ef0cc28&amp;chksm&#x3D;9b1fea57ac6863418063de447915c53d8094ea57e3056d063b24a7e8f434b1d903b14cda1e7e&amp;scene&#x3D;21#wechat_redirect)</span><br><span class="line"></span><br><span class="line">[Linux系统常用命令速查手册](http:&#x2F;&#x2F;mp.weixin.qq.com&#x2F;s?__biz&#x3D;MzAwNTM5Njk3Mw&#x3D;&#x3D;&amp;mid&#x3D;2247494305&amp;idx&#x3D;1&amp;sn&#x3D;35ed91a64a80571d138b016c5f42b11e&amp;chksm&#x3D;9b1fea23ac686335a9e0d9f045af6f79d941a09783e1d5670386aafe05340018373c9de1c938&amp;scene&#x3D;21#wechat_redirect)</span><br><span class="line"></span><br><span class="line">[SpringCloud微服务项目运维必知必会](http:&#x2F;&#x2F;mp.weixin.qq.com&#x2F;s?__biz&#x3D;MzAwNTM5Njk3Mw&#x3D;&#x3D;&amp;mid&#x3D;2247493425&amp;idx&#x3D;1&amp;sn&#x3D;6c0bf357e15af70860c71c66cee43106&amp;chksm&#x3D;9b1fe7b3ac686ea5e7f4780c4d0ef4bf5e4764345991d5a02a34d5a9ac1c8fbb346d75d5f8b5&amp;scene&#x3D;21#wechat_redirect)</span><br><span class="line"></span><br><span class="line">[Python 自动创建 Grafana 仪表板](http:&#x2F;&#x2F;mp.weixin.qq.com&#x2F;s?__biz&#x3D;MzAwNTM5Njk3Mw&#x3D;&#x3D;&amp;mid&#x3D;2247493257&amp;idx&#x3D;1&amp;sn&#x3D;938ac18223d0c3c7e55d217152e29458&amp;chksm&#x3D;9b1fe60bac686f1d000512e4665edb4fcdefec99468ed53659bbfddfad50e2666c5a5e94f9b1&amp;scene&#x3D;21#wechat_redirect)  </span><br><span class="line"></span><br><span class="line">[Shell文本处理三剑客：grep、sed、awk](http:&#x2F;&#x2F;mp.weixin.qq.com&#x2F;s?__biz&#x3D;MzAwNTM5Njk3Mw&#x3D;&#x3D;&amp;mid&#x3D;2247492280&amp;idx&#x3D;1&amp;sn&#x3D;40aa2a9dbb996ccb6eb09c4ee95419ff&amp;chksm&#x3D;9b1fe23aac686b2c6395bb4fa2d7636cb9381e7498fa5ec01936ded503120263db446fcb816c&amp;scene&#x3D;21#wechat_redirect)</span><br><span class="line"></span><br><span class="line">[搭建一套高可用的Harbor容器镜像仓库](http:&#x2F;&#x2F;mp.weixin.qq.com&#x2F;s?__biz&#x3D;MzAwNTM5Njk3Mw&#x3D;&#x3D;&amp;mid&#x3D;2247494266&amp;idx&#x3D;1&amp;sn&#x3D;51434ab592d8010ec8d90c2664dbe922&amp;chksm&#x3D;9b1feaf8ac6863ee0229cfface65c710dc76045c2da248445639563e4b687014c8649745acfe&amp;scene&#x3D;21#wechat_redirect)</span><br><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure>

<p><a target="_blank" rel="noopener" href="http://mp.weixin.qq.com/s?__biz=MzAwNTM5Njk3Mw==&mid=2247491271&idx=1&sn=adcd0c5b32ce81d49aaf15c150133954&chksm=9b1c1e45ac6b97536d61299ca3bfc00ab1db7f589101aee1d86ff74304c61f73921920cb8796&scene=21#wechat_redirect">31天Kubernetes集训营，拿下CKA全球认证！</a><img src=""></p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure>




<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">  </span><br><span class="line"></span><br></pre></td></tr></table></figure>


<p><img src=""></p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">  </span><br><span class="line"></span><br><span class="line">点亮，服务器三年不宕机![](data:image&#x2F;gif;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVQImWNgYGBgAAAABQABh6FO1AAAAABJRU5ErkJggg&#x3D;&#x3D;)</span><br><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure> 
      <!-- reward -->
      
      <div id="reword-out">
        <div id="reward-btn">
          打赏
        </div>
      </div>
      
    </div>
    

    <!-- copyright -->
    
    <div class="declare">
      <ul class="post-copyright">
        <li>
          <i class="ri-copyright-line"></i>
          <strong>版权声明： </strong>
          
          本博客所有文章除特别声明外，著作权归作者所有。转载请注明出处！
          
        </li>
      </ul>
    </div>
    
    <footer class="article-footer">
       
<div class="share-btn">
      <span class="share-sns share-outer">
        <i class="ri-share-forward-line"></i>
        分享
      </span>
      <div class="share-wrap">
        <i class="arrow"></i>
        <div class="share-icons">
          
          <a class="weibo share-sns" href="javascript:;" data-type="weibo">
            <i class="ri-weibo-fill"></i>
          </a>
          <a class="weixin share-sns wxFab" href="javascript:;" data-type="weixin">
            <i class="ri-wechat-fill"></i>
          </a>
          <a class="qq share-sns" href="javascript:;" data-type="qq">
            <i class="ri-qq-fill"></i>
          </a>
          <a class="douban share-sns" href="javascript:;" data-type="douban">
            <i class="ri-douban-line"></i>
          </a>
          <!-- <a class="qzone share-sns" href="javascript:;" data-type="qzone">
            <i class="icon icon-qzone"></i>
          </a> -->
          
          <a class="facebook share-sns" href="javascript:;" data-type="facebook">
            <i class="ri-facebook-circle-fill"></i>
          </a>
          <a class="twitter share-sns" href="javascript:;" data-type="twitter">
            <i class="ri-twitter-fill"></i>
          </a>
          <a class="google share-sns" href="javascript:;" data-type="google">
            <i class="ri-google-fill"></i>
          </a>
        </div>
      </div>
</div>

<div class="wx-share-modal">
    <a class="modal-close" href="javascript:;"><i class="ri-close-circle-line"></i></a>
    <p>扫一扫，分享到微信</p>
    <div class="wx-qrcode">
      <img src="//api.qrserver.com/v1/create-qr-code/?size=150x150&data=http://example.com/2020/11/11/k8s/kubernetes%20%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98%E6%80%BB%E7%BB%93/" alt="微信分享二维码">
    </div>
</div>

<div id="share-mask"></div>  
  <ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/k8s/" rel="tag">k8s</a></li></ul>

    </footer>
  </div>

   
  <nav class="article-nav">
    
      <a href="/2020/11/11/k8s/Kubernetes%20%E6%96%B0%E7%8E%A9%E6%B3%95%EF%BC%9A%E5%9C%A8%20yaml%20%E4%B8%AD%E7%BC%96%E7%A8%8B/" class="article-nav-link">
        <strong class="article-nav-caption">上一篇</strong>
        <div class="article-nav-title">
          
            Kubernetes 新玩法：在 yaml 中编程.md
          
        </div>
      </a>
    
    
      <a href="/2020/11/11/k8s/%E4%BD%BF%E7%94%A8Kubeadm%E8%AE%BE%E7%BD%AEKubernetes%E9%9B%86%E7%BE%A4%E7%9A%84%E9%AB%98%E5%8F%AF%E7%94%A8%E6%80%A7/" class="article-nav-link">
        <strong class="article-nav-caption">下一篇</strong>
        <div class="article-nav-title">使用Kubeadm设置Kubernetes集群的高可用性.md</div>
      </a>
    
  </nav>

   
<!-- valine评论 -->
<div id="vcomments-box">
  <div id="vcomments"></div>
</div>
<script src="//cdn1.lncld.net/static/js/3.0.4/av-min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/valine@1.4.14/dist/Valine.min.js"></script>
<script>
  new Valine({
    el: "#vcomments",
    app_id: "",
    app_key: "",
    path: window.location.pathname,
    avatar: "monsterid",
    placeholder: "给我的文章加点评论吧~",
    recordIP: true,
  });
  const infoEle = document.querySelector("#vcomments .info");
  if (infoEle && infoEle.childNodes && infoEle.childNodes.length > 0) {
    infoEle.childNodes.forEach(function (item) {
      item.parentNode.removeChild(item);
    });
  }
</script>
<style>
  #vcomments-box {
    padding: 5px 30px;
  }

  @media screen and (max-width: 800px) {
    #vcomments-box {
      padding: 5px 0px;
    }
  }

  #vcomments-box #vcomments {
    background-color: #fff;
  }

  .v .vlist .vcard .vh {
    padding-right: 20px;
  }

  .v .vlist .vcard {
    padding-left: 10px;
  }
</style>

 
     
</article>

</section>
      <footer class="footer">
  <div class="outer">
    <ul>
      <li>
        Copyrights &copy;
        2015-2020
        <i class="ri-heart-fill heart_icon"></i> TzWind
      </li>
    </ul>
    <ul>
      <li>
        
        
        
        由 <a href="https://hexo.io" target="_blank">Hexo</a> 强力驱动
        <span class="division">|</span>
        主题 - <a href="https://github.com/Shen-Yu/hexo-theme-ayer" target="_blank">Ayer</a>
        
      </li>
    </ul>
    <ul>
      <li>
        
        
        <span>
  <span><i class="ri-user-3-fill"></i>访问人数:<span id="busuanzi_value_site_uv"></span></s>
  <span class="division">|</span>
  <span><i class="ri-eye-fill"></i>浏览次数:<span id="busuanzi_value_page_pv"></span></span>
</span>
        
      </li>
    </ul>
    <ul>
      
    </ul>
    <ul>
      
    </ul>
    <ul>
      <li>
        <!-- cnzz统计 -->
        
        <script type="text/javascript" src='https://s9.cnzz.com/z_stat.php?id=1278069914&amp;web_id=1278069914'></script>
        
      </li>
    </ul>
  </div>
</footer>
      <div class="float_btns">
        <div class="totop" id="totop">
  <i class="ri-arrow-up-line"></i>
</div>

<div class="todark" id="todark">
  <i class="ri-moon-line"></i>
</div>

      </div>
    </main>
    <aside class="sidebar on">
      <button class="navbar-toggle"></button>
<nav class="navbar">
  
  <div class="logo">
    <a href="/"><img src="/images/ayer-side.svg" alt="Hexo"></a>
  </div>
  
  <ul class="nav nav-main">
    
    <li class="nav-item">
      <a class="nav-item-link" href="/">主页</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/archives">归档</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/categories">分类</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/tags">标签</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" target="_blank" rel="noopener" href="http://www.baidu.com">百度</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/friends">友链</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/2019/about">关于我</a>
    </li>
    
  </ul>
</nav>
<nav class="navbar navbar-bottom">
  <ul class="nav">
    <li class="nav-item">
      
      <a class="nav-item-link nav-item-search"  title="搜索">
        <i class="ri-search-line"></i>
      </a>
      
      
      <a class="nav-item-link" target="_blank" href="/atom.xml" title="RSS Feed">
        <i class="ri-rss-line"></i>
      </a>
      
    </li>
  </ul>
</nav>
<div class="search-form-wrap">
  <div class="local-search local-search-plugin">
  <input type="search" id="local-search-input" class="local-search-input" placeholder="Search...">
  <div id="local-search-result" class="local-search-result"></div>
</div>
</div>
    </aside>
    <script>
      if (window.matchMedia("(max-width: 768px)").matches) {
        document.querySelector('.content').classList.remove('on');
        document.querySelector('.sidebar').classList.remove('on');
      }
    </script>
    <div id="mask"></div>

<!-- #reward -->
<div id="reward">
  <span class="close"><i class="ri-close-line"></i></span>
  <p class="reward-p"><i class="ri-cup-line"></i>请我喝杯咖啡吧~</p>
  <div class="reward-box">
    
    
  </div>
</div>
    
<script src="/js/jquery-2.0.3.min.js"></script>


<script src="/js/lazyload.min.js"></script>

<!-- Tocbot -->


<script src="/js/tocbot.min.js"></script>

<script>
  tocbot.init({
    tocSelector: '.tocbot',
    contentSelector: '.article-entry',
    headingSelector: 'h1, h2, h3, h4, h5, h6',
    hasInnerContainers: true,
    scrollSmooth: true,
    scrollContainer: 'main',
    positionFixedSelector: '.tocbot',
    positionFixedClass: 'is-position-fixed',
    fixedSidebarOffset: 'auto'
  });
</script>

<script src="https://cdn.jsdelivr.net/npm/jquery-modal@0.9.2/jquery.modal.min.js"></script>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/jquery-modal@0.9.2/jquery.modal.min.css">
<script src="https://cdn.jsdelivr.net/npm/justifiedGallery@3.7.0/dist/js/jquery.justifiedGallery.min.js"></script>

<script src="/dist/main.js"></script>

<!-- ImageViewer -->

<!-- Root element of PhotoSwipe. Must have class pswp. -->
<div class="pswp" tabindex="-1" role="dialog" aria-hidden="true">

    <!-- Background of PhotoSwipe. 
         It's a separate element as animating opacity is faster than rgba(). -->
    <div class="pswp__bg"></div>

    <!-- Slides wrapper with overflow:hidden. -->
    <div class="pswp__scroll-wrap">

        <!-- Container that holds slides. 
            PhotoSwipe keeps only 3 of them in the DOM to save memory.
            Don't modify these 3 pswp__item elements, data is added later on. -->
        <div class="pswp__container">
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
        </div>

        <!-- Default (PhotoSwipeUI_Default) interface on top of sliding area. Can be changed. -->
        <div class="pswp__ui pswp__ui--hidden">

            <div class="pswp__top-bar">

                <!--  Controls are self-explanatory. Order can be changed. -->

                <div class="pswp__counter"></div>

                <button class="pswp__button pswp__button--close" title="Close (Esc)"></button>

                <button class="pswp__button pswp__button--share" style="display:none" title="Share"></button>

                <button class="pswp__button pswp__button--fs" title="Toggle fullscreen"></button>

                <button class="pswp__button pswp__button--zoom" title="Zoom in/out"></button>

                <!-- Preloader demo http://codepen.io/dimsemenov/pen/yyBWoR -->
                <!-- element will get class pswp__preloader--active when preloader is running -->
                <div class="pswp__preloader">
                    <div class="pswp__preloader__icn">
                        <div class="pswp__preloader__cut">
                            <div class="pswp__preloader__donut"></div>
                        </div>
                    </div>
                </div>
            </div>

            <div class="pswp__share-modal pswp__share-modal--hidden pswp__single-tap">
                <div class="pswp__share-tooltip"></div>
            </div>

            <button class="pswp__button pswp__button--arrow--left" title="Previous (arrow left)">
            </button>

            <button class="pswp__button pswp__button--arrow--right" title="Next (arrow right)">
            </button>

            <div class="pswp__caption">
                <div class="pswp__caption__center"></div>
            </div>

        </div>

    </div>

</div>

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/default-skin/default-skin.min.css">
<script src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe-ui-default.min.js"></script>

<script>
    function viewer_init() {
        let pswpElement = document.querySelectorAll('.pswp')[0];
        let $imgArr = document.querySelectorAll(('.article-entry img:not(.reward-img)'))

        $imgArr.forEach(($em, i) => {
            $em.onclick = () => {
                // slider展开状态
                // todo: 这样不好，后面改成状态
                if (document.querySelector('.left-col.show')) return
                let items = []
                $imgArr.forEach(($em2, i2) => {
                    let img = $em2.getAttribute('data-idx', i2)
                    let src = $em2.getAttribute('data-target') || $em2.getAttribute('src')
                    let title = $em2.getAttribute('alt')
                    // 获得原图尺寸
                    const image = new Image()
                    image.src = src
                    items.push({
                        src: src,
                        w: image.width || $em2.width,
                        h: image.height || $em2.height,
                        title: title
                    })
                })
                var gallery = new PhotoSwipe(pswpElement, PhotoSwipeUI_Default, items, {
                    index: parseInt(i)
                });
                gallery.init()
            }
        })
    }
    viewer_init()
</script>

<!-- MathJax -->

<!-- Katex -->

<!-- busuanzi  -->


<script src="/js/busuanzi-2.3.pure.min.js"></script>


<!-- ClickLove -->

<!-- ClickBoom1 -->

<!-- ClickBoom2 -->

<!-- CodeCopy -->


<link rel="stylesheet" href="/css/clipboard.css">

<script src="https://cdn.jsdelivr.net/npm/clipboard@2/dist/clipboard.min.js"></script>
<script>
  function wait(callback, seconds) {
    var timelag = null;
    timelag = window.setTimeout(callback, seconds);
  }
  !function (e, t, a) {
    var initCopyCode = function(){
      var copyHtml = '';
      copyHtml += '<button class="btn-copy" data-clipboard-snippet="">';
      copyHtml += '<i class="ri-file-copy-2-line"></i><span>COPY</span>';
      copyHtml += '</button>';
      $(".highlight .code pre").before(copyHtml);
      $(".article pre code").before(copyHtml);
      var clipboard = new ClipboardJS('.btn-copy', {
        target: function(trigger) {
          return trigger.nextElementSibling;
        }
      });
      clipboard.on('success', function(e) {
        let $btn = $(e.trigger);
        $btn.addClass('copied');
        let $icon = $($btn.find('i'));
        $icon.removeClass('ri-file-copy-2-line');
        $icon.addClass('ri-checkbox-circle-line');
        let $span = $($btn.find('span'));
        $span[0].innerText = 'COPIED';
        
        wait(function () { // 等待两秒钟后恢复
          $icon.removeClass('ri-checkbox-circle-line');
          $icon.addClass('ri-file-copy-2-line');
          $span[0].innerText = 'COPY';
        }, 2000);
      });
      clipboard.on('error', function(e) {
        e.clearSelection();
        let $btn = $(e.trigger);
        $btn.addClass('copy-failed');
        let $icon = $($btn.find('i'));
        $icon.removeClass('ri-file-copy-2-line');
        $icon.addClass('ri-time-line');
        let $span = $($btn.find('span'));
        $span[0].innerText = 'COPY FAILED';
        
        wait(function () { // 等待两秒钟后恢复
          $icon.removeClass('ri-time-line');
          $icon.addClass('ri-file-copy-2-line');
          $span[0].innerText = 'COPY';
        }, 2000);
      });
    }
    initCopyCode();
  }(window, document);
</script>


<!-- CanvasBackground -->


    
  </div>
</body>

</html>